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Abstract 

This  paper  unifies  the  development  of  the  cutting  plane  algorithm  for 
mathematical  programs  and  variational  inequalities  by  providing  one 
common  framework  for  establishing  convergence.  Strategies  for  generating 
cuts  are  provided  for  cases  in  which  the  algorithm  yields  easy  and  difficult 
subproblems.  When  the  subproblem  is  easy  to  solve,  a  line  search  is  added 
and  a  deep  cut  is  selected  to  accelerate  the  algorithm.  On  the  other  hand, 
when  the  subproblem  is  difficult  to  solve,  the  problem  is  only  solved 
approximately  during  the  early  iterations.  This  corresponds  to  generating 
cuts  which  are  nontangential  to  the  underlying  objective  function.  Moreover, 
in  the  case  of  variational  inequalities,  it  is  shown  further  that  the  subproblem 
can  be  eliminated  entirely  from  the  algorithmic  steps,  thereby  making  the 
resulting  algorithm  especially  advantageous. 


I.      INTRODUCTION 

Many  have  studied  the  cutting  plane  approach  for  mathematical 
programs  for  some  time.  Cheney  and  Goldstein  (1959)  and  Kelly  (1960)  first 
proposed  it  for  convex  programs.  Dantzig-Wolfe  (1960)  (see  also  Dantzig, 
1963)  proposed  the  dual  equivalent  called  generalized  linear  programming  or 
column  generation.  Zangwill  (1969)  provided  new  viewpoints  and  supplied 
a  convergence  proof  based  on  algorithmic  maps.  Magnanti  et  al.  (1976) 
showed  that  generalized  linear  programming  solves  Lagrangian  dual 
problems  even  when  standard  convexity  assumptions  do  not  hold.  As  a 
standard  technique,  the  cutting  plane  approach  often  appears  in  texts  such  as 
Bazaraa  and  Shetty  (1979). 

For  variational  inequalities,  references  on  the  cutting  plane  approach  are 
considerably  less.  Zuhovickii  et  al.  (1969)  (see  also  Auslender,  1976)  were  the 
first  to  apply  it  to  variational  inequalities.  Nguyen  and  Dupuis  (1984) 
proposed  an  acceleration  idea  and  prove  convergence  using  Zangwill's 
algorithmic  maps. 

It  is  well  known  (see,  e.g.,  Auslender,  1976,  Hearn  et  al.,  1984  and  Nguyen 
and  Dupuis,  1984)  that  mathematical  programs  (MPs)  and  variational 
inequalities  (Vis)  are  related  and  possess  many  similar  properties.  In  fact, 
many  algorithms  for  VI  problems  are  based  on  MP  algorithms  (see,  Harker 
and  Pang,  1990).  In  spite  of  this  relationship,  the  developments  of  the  cutting 
plane  algorithm  appear  different  for  these  two  areas.  However,  one  can  unify 
the  developments  by  addressing  MPs  and  Vis  in  a  common  framework.  In 
this  paper,  we  provide  one  such  framework  with  which  the  cutting  plane 
algorithm  can  be  derived  for  both  MPs  and  Vis.  This  framework  is  then  used 


to  analyze  existing  schemes  for  accelerating  the  cutting  plane  algorithm.  The 
analysis  results  in  a  common  argument  for  proving  convergence  for  all 
acceleration  schemes  regardless  of  the  underlying  problems  and  points  out 
when  each  scheme  might  be  effective. 

For  MPs,  this  paper  focuses  on  problems  derived  from  Benders 
decomposition  and  Lagrangian  duality.  For  Vis,  the  paper  considers  those 
with  finite  dimension.  In  Section  2,  it  is  shown  that  these  problems  can  be 
stated  as  the  following  maximin  problem: 

w*  =  max  min  {f(x)  +  ug[x)} 
ueU  x  eX 

where  U  and  X  are  nonempty  and  convex  subsets  of  Rm  and  Rn,  respectively, 
f(x)  is  a  continuous  real-valued  function  define  on  X,  g(x)  is  a  continuous 
vector-valued  function  mapping  R"  into  Rm,  and  w*  is  the  maximin  value  for 
the  problem.  For  convenience,  ug(x)  denotes  tne  dot  product  of  vectors  u  and 
g(x).  The  basic  cutting  plane  algorithm  is  stated  in  Section  3  with  a 
convergence  proof.  Section  4  utilizes  the  framework  given  in  Section  3  to 
analyze  existing  strategies  for  accelerating  the  algorithm.  Finally,  Section  5 
concludes  the  paper. 


2.     INSTANCES  OF  MAXIMIN  OPTIMIZATION 

The  first  two  instances  of  the  maximin  problem  were  first  observed  in 
Magnanti  and  Wong  (1981)  and  they  are  stated  here  for  completeness.  The  last 
instance  can  be  found  in  a  slightly  different  form  in  Hearn  et  al.  (1984)  and 
Nguyen  and  Dupuis  (1984). 

First,  consider  the  following  nonlinear  program  (NLP) 

min  f(x) 
s.t.    g(x)<0 
x  e  X 

where  f(x),  g{x),  and  X  are  as  previously  defined  with  the  exception  that  X  is 
additionally  assumed  to  be  compact.  Then,  the  Lagrangian  dual  problem  of 
the  NLP  can  be  stated  as 

max  min  \f(x)  +  ug{x)} 

u>0  xeX 

where  u  now  represents  the  dual  vector  for  the  constraints  defined  by  g(x). 
This  shows  that  Lagrangian  duality  leads  to  the  desired  maximin  problem. 

When  f(x)  =  ex,  g(x)  =  b-Ax,  and  X  =  {x:  Dx  >  d,  x  >  0}.  The  above  NLP 
reduces  to  the  following  linear  program  (LP): 

min    ex 
s.t.      Ax  >  b 
Dx>d 
x  >0, 

and  the  Lagrangian  dual  of  this  LP  is 


max  min  {ex  +  a(b  -  Ax)} 

«>0    xaX 


Letting  f{x)  =  ex  and  g(x)  =  b-Ax  yields  the  desired  maximin  problem. 
However,  an  equivalent  method  of  obtaining  the  Lagrangian  dual  of  the  LP  is 
by  applying  Benders  decomposition  to  the  dual  of  the  LP  which  can  be  stated 
as 

max    ub+vd 
s.t.      uA+vD<c 
u,v>0. 

Partitioning  the  decision  variables  (u,v)  gives 

max  max  {ub+  vd:  vD  <  c  -  uA}. 
u  >0    i>>0 

By  dualizing  the  inner  maximization,  the  above  maximin  problem  can  be 
written  as 

max  m in  {ub  +  (c  ■  uA)x} ,  or 
i/>0    xcx 

max min  {ex  +  n{b  -  Ax)}. 

w>0    xdX 

The  last  maximin  problem  is  the  same  as  the  one  obtained  with  Lagrangian 
duality. 

Our  last  instance  of  a  maximin  problem  is  a  variational  inequality  (VI) 
problem  which  can  be  stated  as  follows:  find  a  vector  x*  €  S  such  that 

F(x*)(u-  x*)>0     Vu  eS 

where  S  is  a  nonempty,  convex  and  compact  subset  of  R"  and  F(x)  is  a  vector 
function  mapping  Rn  into  Rn.  When  F(x)  is  strongly  monotone  on  S,  i.e.,  3  a  > 
0  such  that 

(F(x)  -  F(u))(x  -u)>a\x-u\2  V  x,u  €  S, 


then  the  VI  problem  can  be  stated  as  [see  Hearn  et  al.  (1984)  and  Nguyen  and 
Dupuis  (1984)] 

max  min  {F(x){x  -  u)},   or  max  min  {F(x)x  -  uF(x)}. 
ueS      xeS  u€.S    xeS 

Letting  f(x)  =  F(x)x  and  g(x)  =  -F(x)  again  yields  the  desired  maximin  problem. 


3.     THE  CUTTING  PLANE  ALGORITHM 

In  the  basic  cutting  plane  algorithm,  the  optimal  value  of  the  inner 
minimization  of  the  maximin  problem  at  a  given  point  u  is  denoted  as  Liu), 
i.e., 

[S]:  L(u)  =min{f(x)+ug(x)} 

xeX 

Since  Liu)  is  defined  as  the  pointwise  minimum  of  a  set  of  functions  linear  in 
u,  Liu)  must  be  concave.    To  motivate  the  algorithm,  the  maximin  problem  is 
restated  as 
[P]:  W*  =  max      w 

s.t.       w  <f(x?)  +  ug(x!)     VxJ  €  X 

u  €  U  and  w  is  unrestricted. 

Problem  [P]  has  an  infinite  number  of  constraints  of  the  form 

zv<f(x^)  +  ug(x) 

which  are  generally  referred  to  as  cutting  planes  or  simply  cuts.  To  avoid 
generating  all  the  cuts  apriori,  the  algorithm  initially  solves  an  approximation 
of  problem  [P]  which  contains  only  a  few  cuts  and  obtain,  say  (zv',u'),  as  a 
solution.  To  further  refine  the  approximation  and  hence  obtain  a  more 
accurate  solution,  problem  [S]  is  solved  with  u  =  u'  to  produce  a  solution  x' 
which  defines  a  new  cut 

w  <f(x')  +  ug(x'). 


The  approximation  of  problem  [P]  is  updated  and  resolved  with  the  addition 
of  this  new  cut.  Then,  the  process  is  repeated  until  an  optimal  solution  is 
found.  Below  we  formally  state  the  algorithm  for  the  maximin  problem. 

The  Cutting  Plane  (CP)  Algorithm 

Step  0:   Find  a  point  x°  e  X.  Set  k  =  1  and  go  to  Step  1. 
Step  1:   Solve  the  (k^  )  master  problem 

max        w 
s.t.       w  <  f(xl)  +  ug(x{)    i  =  0, ...,  k-l 
u  £  U  and  zu  unrestricted. 

Let  {wk,  uk)  denote  an  optimal  solution  and  go  to  Step  2. 

Step  2:   Solve  the  (k^  )  subproblem 

xk  =  argminl  f{x) +  ukg{x)} 

and  L(uk)  =  f[xk)  +  ukg(xk). 

Step  3:   If  zv k  =  L(nk),  then  (nk,  xk)  solves  the  maximin  problem. 

Otherwise  (i.e.,  wk  >  L(uk)),  replace  k  by  k+l  and  go  to  Step  1. 

In  Step  1,  the  dual  of  the  kth  master  problem  is  the  following  linear  pro- 
gram 

Jt-l 

min       X   Kjfix1) 

i=0 

Jt-l 

S-t.  I    7tj  g(xl)     <  0 

i=0 

k-l 

In,        =1 
/=0 

nt       >  0       Vi-0,—,(Jfc  -1) 


The  CP  algorithm  with  the  master  problem  replaced  by  its  dual  as  stated 
above  is  generally  known  as  Dantzig-Wolfe  decomposition,  column  genera- 
tion, or  generalized  linear  programming. 

In  both  Benders  decomposition  and  Lagrangian  dual  of  an  NLP,  the  set  X 
is  either  constructed  or  chosen  to  facilitate  the  solving  of  the  subproblem  in 
Step  2  or,  equivalently,  problem  [S].  In  many  cases,  a  closed  form  solution  or 
efficient  algorithm  exists  for  the  subproblem.  However,  such  is  not  the  case 
for  Vis.  The  choice  for  X  is  restricted  to  be  S,  the  feasible  region  of  the  VI 
problem.  Moreover,  the  objective  function  of  the  subproblem  contains  the 
term  F{x)x  which  is  at  least  quadratic,  unless  Fix)  is  a  constant  vector. 
Therefore,  the  subproblem,  or  problem  [S],  does  not  generally  admit  an  easy 
solution  and  the  CP  algorithm  as  stated  above  would  be  inefficient,  if  not 
ineffective,  for  Vis.  In  Section  4,  a  technique  to  overcome  this  difficulty  is 
discussed. 

Step  3  uses  the  fact  that  wk  >  L(uk)  which  is  generally  known  to  follow  from 
Lagrangian  duality.  However,  the  inequality  can  also  be  obtained  by  noting 
that  x)  e  X  for  all  j  and 


L(uk)  =  min{/(x)  +  ukg(x)}  <  min{/(x;)  +  u*g(x;):0  <  ;'  <  k  - 1}  =  w 


So,  regardless  of  how/(x)  and  g(x)  are  derived,  the  inequality  holds. 

The  following  two  lemmas  are  essential  for  the  convergence  of  the  CP 
algorithm.  The  proofs  of  both  lemmas  are  similar  to  those  in  Dantzig  (1963) 
and  Magnanti  et  al.  (1976)  with  the  exception  that  all  references  to  Lagrangian 
duality  are  eliminated.  However,  they  are  stated  here  for  completeness  and 
further  reference. 


Lemma  1:   At  each  iteration  of  the  CP  algorithm 

(i)     w^1  >wk      V  k 

(ii)    ivk>w*        V  k 

(iii)  If  L(uk)  >zvk,  then  zv*  -  L(uk)  =  wk. 
Proof:  The  first  result  follows  from  the  fact  that  the  /cth  master  problem  has 
more  cuts  than  the  (k-l)st.  The  second  follows  from  the  fact  that  the  master 
problem  in  Step  1  contains  only  a  finite  subset  of  cuts  in  problem  [P]. 
If  L(uk)  >  wk,  we  have 

zv  *  <  wk 

<L(uk)  =min{  f(x) +  ukg(x)} 

<  max  minif(x)  +  ug(x)}  =  zv  * 

ueU   xeX 

where  the  first  inequality  follows  from  (ii),  the  second  is  given  in  (iii),  and  the 
last  inequality  is  because  of  the  maximization  over  the  set  U.  The  above  series 
of  inequalities  implies  that  (iii)  holds.  □ 

Lemma  2:    If  X  is  compact  and  there  exists  a  subset  K  Q  [1,  2,  ...}  such  that  the 
subsequence  {uk}keK  is  convergent,  say  to  the  limit  point,  u°°,  then 

limw    =  w*  =  tilt00). 
keK  K      ' 

Proof:  Since  (wk,  uk)  solves  the  kth  master  problem  and  the  fact  that  constraints 

in  the  ;th  master  problem,  with  ;  <  k,  are  always  contained  in  the  problem  at 

iteration  k,  we  have  that 

f(xj)  +  ukg(xj)>wk  >w*  for;  =0,l,...,(fc  -  1)  (1) 

where  the  last  inequality  follows  from  Lemma  1.  Let 


w     =  limit;  . 
keK 

Note  that  w  °°  exists  because  {u'^isa  monotonically  decreasing  sequence 
which  is  bounded  below  by  w*.  Taking  the  limit  in  equation  (1)  for  k  £  K,  we 
obtain 


f[xj)  +  u°°g{xj)  >iv°°>w*      for;  =  0,1,2,3, ... 


(2) 


By  assumption,  g{x)  is  continuous  and  X  is  compact.   There  must  exist  a  posi- 
tive real  number  j3  such  that  |  g(x)  |  <  /3  for  all  x  £  X.  Then, 

\f(xk)  *  ukg(xk)  ■  /(/)  -  u-g(xk)\  =  |(,/'  ■  «- )g{x*)\  <  ji\uk  ■  »'  \  (3) 

Consequently,  for  any  given  e  >  0,  there  is  a  k\  £  K  such  that  for  all  k  £  K  and 
k  >  k\,  the  last  term  in  (3)  is  bounded  by  e.  Therefore, 

f(xk)  +  ukg(xk)-f(xk)-u~g(xk) 


<£ 


or 


£>f[x')+ll  g[x    )  -  f(x    )  -  U°°g(x  ')>■£. 


Examining  the  right  inequality,  we  obtain 

L(uk)=f(xk)  +  ukg(xk)>f{xk)+u~g{xk)-e 


or 


L(uk)+eZf(xk)  +  u~g(xk). 


From  (2)  and  the  definition  of  w*, 

k\   ,       .   r/    k 


w 


+  e  >  L(«* )  +  e  >  f(xk)  +  u°°g(xk)  >w°°> 


W 


Since  £  is  arbitrary,  we  can  conclude  that 
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tv*  =  \imL\u   )  =  L 
keK    v     ' 


,.       k 
limw 

LJteK 


=  limzt; 
keK 


where  the  middle  equality  follows  from  the  continuity  of  L(u).  □ 

To  obtain  a  solution  to  the  maximin  problem,  evaluate  the  function  Liu) 
at  u°°  to  produce  x°°  ,i.e., 

x°°  =  argmin{/(x)  +  u°g(x)}. 

Then,  it  follows  from  Lemma  2  that  (x  ,u  )  solves  the  maximin  problem. 

We  now  obtain  convergence  for  the  three  instances  of  maximin  problems. 

For  Benders  decomposition  of  an  LP,  the  second  condition  in  Lemma  1  will 

hold  after  a  finite  number  of  iterations  because  the  set  X  is  assumed  to  be 

compact  and  can  be  expressed  as  a  convex  combination  of  a  finite  number  of 

extreme  points  [see,  e.g.,  Bazaraa  et  al.  (1990)].  For  VI  problems,  both  U  and  X 
are  the  same  as  S  which  is  compact.  Since  uk  €  S  for  all  k,  {uk}k  must  contain  a 

convergent  subsequence,  thereby  satisfying  the  hypothesis  of  Lemma  2.  So, 
the  CP  algorithm  converges  for  Vis.  The  convergence  for  the  Lagrangian  dual 
of  an  NLP  follows  from  the  theorem  below. 

Theorem  3:    If  there  exists  an  x°  €  X  such  that  g(x°)  <  0  then  there  is  a 
converging  subsequence  of  {uk}k  . 

Proof:  See  Fisher  and  Shapiro  (1974). 

This  theorem  also  points  out  that  the  selection  of  the  initial  solution  x°in 
Step  0  of  the  CP  algorithm  is  critical  for  the  Lagrangian  dual  case,  for  it  deter- 
mines the  convergence  of  the  algorithm.  Magnanti  et  al.  (1976)  provides  a 
procedure  to  obtain  an  initial  solution  satisfying  the  condition  in  Theorem  3  if 
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one  is  not  readily  available.   This  procedure  is  akin  to  phase  one  of  the  two 
phase  method  in  LP. 
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4.  STRATEGIES  FOR  GENERATING  CUTS 

The  two  main  steps  in  the  CP  algorithm  consist  of  solving  the  master 
problem  and  generating  a  new  cut.  In  this  section,  we  examine  existing 
schemes  for  generating  new  cuts. 

Solving  the  subproblem  in  Step  2  of  the  CP  algorithm  is  one  method  of 
generating  new  cuts.  However,  it  does  not  distinguish  between  easy  and 
difficult  subproblems.  When  the  subproblem  is  easy  to  solve,  it  would  be 
advantageous  to  solve  more  subproblems  in  a  effort  to  obtain  better  cuts,  i.e., 
those  which  may  lead  to  a  reduction  in  the  number  of  master  problems  to  be 
solved.  On  the  other  hand,  when  the  subproblem  is  difficult  to  solve,  it  may 
be  better  to  cheaply  obtain  a  legitimate  cut,  perhaps  not  necessarily  tangential 
to  Liu).  The  two  subsections  below  discuss  these  two  schemes  in  detail. 

4.1:  Easy  Subproblems 

Hearn  and  Lawphongpanich  (1989a)  viewed  the  difference  of  two 
successive  iterates  of  the  CP  algorithm  as  a  direction,  i.e.,  d  =  uk-uk~l .  They 
showed  that,  when  Liu)  is  differentiable  at  ift'l,  d  is  an  ascent  direction. 
Otherwise,  the  nondifferentiability  at  uk'l  implies  that  the  subproblem  at  the 
(k-l)st  iteration  has  multiple  solutions  and  the  ascent  property  would  depend 
on  the  choice  of  x^-1.  In  any  case,  this  observation  suggests  the  inclusion  of  a 
line  search  step.  Below  is  one  version  of  the  CP  algorithm  with  line  search. 
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The  Cutting  Plane  Algorithm  with  Line  Search  (CPLS) 

Step  0:   Find  a  point  x°  €  X.  Let  v°  =  0  and  k  =  1.  Go  to  Step  1. 

Step  1:  same  as  before. 

Step  2:   same  as  before. 

Step  3:   If  wk  =  L(uk  ),  then  (xk,  uk)  solves  the  maximin  problem.  Otherwise  (i.e., 

wk  >  L(uk)),  set  dk  =  uk-vk~^. 

(i)     If  k  =  1;  set  i?*  =  m*  and  fc  =  ifc+l.  Go  to  Step  1. 

(ii)    Otherwise  (k  >  1),  let 

tmax  =  arg  max  [L(vk-*  +  tdk):0  <  t  <  tup} 

where  tup  is  the  maximum  value  of  f  for  which  vk~l  +  tdk  remains 
feasible  to  U.   If  tmax  <  1,  then  pick  any  nonzero  tk  £  [fmax,l]. 
Otherwise,  let  tk  €  [l,rmax]-    Set  vk  -  vk~l  +  tkdk  and  solve  the 
(sub)problem 

y    =argmin{/(x)+  v  g(v)\. 
If  yk  is  not  unique,  select  yk  £  X(vk)  so  that 

f(yk)  +  »ks{yk)sf{yk)  +  Jz(yk)'L(vk)  '4» 

Set  xk  =  yk  and  k  =  k  +  l.  Go  to  Step  1. 

Note  that  CPLS  produces  two  sets  of  dual  iterates,  uk  and  vk,  where  uk 
denotes  the  solution  of  the  master  problem  and  vk  is  a  point  along  the 
direction  dk  =  uk  -  vk~*.  Moreover,  the  cuts  are  generated  at  vk  instead  of  uk. 

In  Step  3(ii),  the  choice  of  step  length,  tk,  in  the  direction  dk  is  inexact  to 
allow  for  inaccuracy  in  the  line  search  and  for  heuristic  selection  of  a  new  cut. 
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However,  setting  tk  =  tmax  places  vk  at  the  maximum  point  and  tk  ==  1  places  vk 
at  uk  which  reduces  CPLS  to  the  basic  CP  algorithm. 

For  any  allowable  choice  of  tk,  the  resulting  vk  is  feasible  to  the  master 
problem  and  we  have  the  following  relationship 

L(uk)  <  L(vk)  <  min{  fix1)  +  vkg(xl)  :  i  =  0,...,(k-l)}  <  wk 
If  L(vk)  =  wk,  then  CPLS  would  discover  that  (xk,  vk)  is  optimal  in  the  next 
iteration.  Otherwise  (i.e.,  L(vk)  <  wk),  adding  the  cut 

w  <  f(xk)  +  ug(xk) 
makes  the  point  (ivk,nk)  infeasible  to  the  next  master  problem  thereby  ensuring 
that  the  sequence  zvk  decreases  monotonically. 

To  establish  convergence  for  CPLS,  it  is  assumed  that  either  U  is  compact 
or  L(ii)  satisfies  the  following  condition 

[A]:  lim  L(u  +  id)  =  -°° 


t  —  oo 


for  all  u  G  Li  and  for  all  of  its  direction  of  recession,  d  *  0  (see  page  61  of 

Rockafellar,  1970).   Among  the  three  instances  of  maximin  problems,  U  is 

compact  for  VI  problems  because  U  -  S  and  S  is  compact  by  the  standing 
assumption  in  Section  2.   For  optimization  problems,  U  =  {«■  >  0  :  i  =  1, .  .  . ,  m} 

implies  that  every  component  of  its  direction  of  recession  must  be 
nonnegative.  Then,  the  requirement  (in  Theorem  3)  that  there  exists  an  x°  €  X 
such  that  g(x°)  <  0  further  implies  that  condition  [A]  holds.  To  verify  this, 
note  that  for  any  n  €  U  and  any  direction  of  recession  d  *  0 

L(u  +  td)  =  mm{f(x)  +  (u  +  td)g(x)}<f(x0)  +  ug(x0)  +  tdg(x°). 
xeX 

Since  d  >  0  andg(x0)  <  0,  dg(x°)  <  0  and  taking  the  limit  as  t  approaches  °°  gives 
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hmL(ii  +  td)<f(x0)  +  ug(x°)  +  \im  tdg{x°)=  -«>. 

I  — ♦  oo  f  — ►  oo 

Under  the  assumption  that  either  U  is  compact  or  condition  [A]  holds,  the 
value  for  fmax  in  Step  3  of  CPLS  must  be  bounded.  This  in  turn  implies  that  v 

k 

is  also  bounded  whenever  u    is  bounded  because 

\vk\  =  \vk  -uk  +  uk\  <  \vk  -uk\  +  \uk\<  max{l,  tmax)+\uk\. 
Then,  the  convergence  of  CPLS  follows  from  the  lemma  below. 

Lemma  4:  If  X  is  compact  and  there  exists  an  index  set  K  £  {1,  2,  ...}  such  that 
the  subsequence  {uk}ke.K  converges  to  «°°  then  there  exists  a  subset 
K'  c   K  such  that  {vk}k£K'  converges  to  v°°  and 

lim  w    =  w*  =  L(v°°) 
keK'  v      ' 

Proof:   Using  the  same  argument  as  in  Lemma  2,  it  can  be  shown  that 

f[xi)+u°g(xi)  >w°°>zv*      V;  =  0,1,2,-  (5) 

and  f[xk)  +  ukg(xk)  >  f(xk)  +  u°°g(xk)  ■  e. 

Using  inequality  (4)  in  Step  3(ii),  the  right  expression  is  bounded  above  as 
follows. 

L(vk)  =  f(xk )  +  vkg{xk)  >  f[xk)  +  ukg[xk)  >  f[xk)  +  u°°g(xk )  -  e 

or  L(y*)  +  e  >f(xk)  +  ukg{xk)  >  f(xk)  +  u~g(xk). 

From  (5),  we  have 

.k\  ,   „  -^  r(    k\  ,     k   (    k\  -v  A    k\  ,     ~   /„fc\  x  ~..°° 


a; 


* +£>L(iT)+ £>/(/)  + i<£(x*)  ^/(^j  +  u00^*)  >w°°  >w\ 
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Since  {uk}keK  converges,  u  must  be  bounded  for  all  k  e  K.  It  follows  from 
the  preceding  discussion  that  v  for  allk  €  K  must  be  bounded  as  well.  Thus, 
there  must  exist  a  subset  K'  c  K  such  that  the  subsequence  {vk}keK'  converges 
to  v°°.  So,  taking  the  limit  of  the  above  inequalities  with  respect  to  A:  €  K',  we 
have 


+£>  hmL(vk)  +  £  >L 

k£K'    v      ' 


,.       k 

limy 


+  e  >w*. 


IkeK' 
Since  e  is  arbitrary,  w*  =  L(v°°).  □ 

Given  Lemma  4,  obtaining  the  x  component  for  the  maximin  solution 
corresponding  to  v°°  and  establishing  the  convergence  of  the  three  instances 
are  the  same  as  in  the  basic  CP  algorithm. 

At  the  end  of  Step  3(ii),  xk  originally  defined  in  Step  2  is  replaced  with  yk. 
So,  yk  now  defines  a  new  cut  for  the  (A:+2)st  master  problem.  Moreover,  yk  is 
not  arbitrarily  selected  when  alternate  optima  exist.  In  fact,  Step  3  requires 
that  yk  satisfies  inequality  (4)  and  the  theorem  below  ensures  that  such  yk 
exists. 

Theorem  5:    In  Step  3  of  CPLS,  there  exists  a  yk  which  solves 

L(v    )  =  minf/(x)  +  v  g(x)\ 
xeX  L  J 

and  satisfies 

f(yk)  +  ukg(yk)  <f(yk)  +  vkg(yk)  =  L(vk  ) 

where  nk  is  part  of  the  solution,  (wk/uk),  to  the  fc*  master  problem. 
Proof:   Assume  that   tm av  >  1.   Denote  vma%  =  v      +  tm„d  .  Then,  v  =  B  ymax  + 

UldX  II  lei  X  * 

(1-/3)  u    for  some  /3  e  (0,1)  and  by  concavity  of  L(u) 
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L(vk)  >  /3  L(vmax )  +  (l-P)L(u)  >  Liu). 
Let  Xiv  )  denote  the  (compact)  set  of  solutions  to  the  subproblem.  To  obtain  a 


contradiction,  assume  that 


Then 


which  implies  that 


fix)  +  u  gix)  >f(x)  +  v  g(x)  for  all  x  e  Xiv  ) 


(u    -  v  )  g(x)  >  0  for  all  x  £  X(vk) 


min  {  (u    -  V  )  g(x) :  x  G  X(v  )}  >  0 


Thus,  the  directional  derivative  of  L(v)  at  v    in  the  direction  (u     -  v    )  is 

k        k  k 

positive,  i.e.,  (u    -  v  )  is  an  ascent  direction.   However,  v    is  a  point  on  the  line 
connecting  v      '  to  u   and,  by  the  concavity  of  Liu),  moving  toward  u   must 

k  k 

decrease  its  value.   This  contradicts  the  statement  that  (u    -  v   )  is  an  ascent 
direction.  (The  case  for  tmax  <  1  is  proved  similarly.)  □ 


Figure  1  illustrates  cuts  which  satisfy  or  are  'allowed'  by  inequality  (4).  In 
this  case,  there  are  an  infinite  number  of  allowable  cuts  and  any  of  them 
would  make  CPLS  converges. 

As  in  Magnanti  and  Wong  (1981),  we  say  that  the  cut,  w  <  f(x')+ug(x'), 
dominates  or  is  stronger  than  the  cut,  w  <  f{x")  +  ug(x")  if 

f{x')  +  ug{x')<f(x")+ug(x")     Vu  eU 

with  a  strict  inequality  for  at  least  one  u.  We  call  a  cut  pareto  optimal  if  no 
cut  dominates  it.  Since  a  cut  is  determined  by  x  G  X,  we  also  say  that  x' 
dominates  x"  (or  x'  is  stronger  than  x")  if  the  associated  cut  is  stronger,  and  we 
say  that  x  is  pareto  optimal  if  the  corresponding  cut  is  pareto  optimal. 
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It  is  interesting  to  note  that  every  allowable  cut  in  Figure  1  is  pareto 
optimal.  However,  this  is  not  always  true.  In  general,  there  are  allowable  cuts 
which  are  dominated.  Theorem  6  shows  how  to  modify  Step  3(ii)  of  CPLS  to 
generate  a  pareto  optimal  cut. 


previous  cuts 


an  allowable  cut 


L(u) 


Figure  1.  Allowable  Cuts  for  Step  3  of  CPLS 
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Theorem  6:    Let  X(vk)  denote  the  solution  set  of  the  following  (sub)problem. 

min{f(x)+vkg(x)}. 
xeX l  J 

U  yk  uniquely  solves 

min    lf(x)  +  ukg(x)}, 

x€x(vky  J 

then  yk  is  pareto  optimal. 

Proof:   Assume  the  contrary  that  yk  is  not  pareto  optimal,  i.e.,  there  exists  an 
x  €.  X  and  x  ±yk  such  that 

f{x)+ug(x)<f(yk)+ug{yk)     Vu€U.  (6) 

In  particular,  setting  u  -  v  in  (6)  we  obtain 

f(x)  +  vkg(x)<f(yk)+vkg(yk).  (7) 

Since  yk  €  X(vk), 

f(yk)+vkg(yk)<f(x)+vkg(x).  (8) 

Combining  (7)  and  (8)  produces  the  following  equality 

f{yk)+vkg(yk)=f(x)+vkg(x) 

that  is,  x  €  X(^).  However,  setting  u  -  u   in  (6)  gives 

f{x)  +  ukg(x)<Lf(yk)+ukg(yk). 

Since  x  ±yk  and  yk  uniquely  solves  the  2nd  minimization  problem,  this  last 
inequality  is  a  contradiction.  □ 
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If  uk  in  Theorem  6  is  in  the  relative  interior  of  U ,  the  uniqueness 
assumption  for  yk  can  be  dropped  and  the  pareto  optimality  follows  from 
Theorem  1  in  Magnanti  and  Wong  (1981).  The  following  theorem  shows  that 
yk  in  Theorem  6  defines  an  allowable  cut. 

Theorem  7:  Let  yk  be  as  defined  in  Theorem  6.  Then 

f(yk)  +  ukg(yk)<f(yk)  +  vkg(yk) 

Proof:  Assume  the  contrary  that 

f(yk)  +  nkg(yk)  >fiyk)  +  vkgiyk)- 

Using  the  definition  of  yk,  we  have  that 

min   {/(y)  +  u*g{y)}  >/(/)  +  vkg{yk) 

min  {f(y)  +  ukg(y)}-f{yk)-vkg(yk)>o. 

Since  f(yk)  +  vkg(yk)  =  L(vk)  =  f(y)  +  vkg(y)  for  all  y  e  X(vk), 

min  {f[y)  +  ukg(y)-f(y)-vkg(y)}>0. 

yex(vk) 

min   {(uk  -  vk)g(y)}  >  0. 


The  expression  on  the  left  side  is  the  directional  derivative  of  L{vk)  in  the 
direction  (uk-vk),  and  the  inequality  implies  that  uk-vk  is  an  ascent  direction 
which  is  impossible  for  our  choice  of  step  size  tk.  To  see  this,  assume  that  tk  G 
lAnax/1]-  This  implies  that  vk  is  on  line  joining  v™'*  and  nk,  where  vm'x  =  z>;'"2   + 
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^max  (uk-vk)  maximizes  L(u)  in  the  direction  (uk-vk).  So,  moving  away  from  ifi 
toward  uk  can  only  decrease  the  function  value,  i.e.,  uk-vk  cannot  be  an  ascent 
direction  at  p*.  The  argument  is  the  same  for  tk  e  [l,fmax].  D 

Theorems  6  and  7  demonstrate  that  it  is  possible  to  select  an  allowable  cut 
which  is  also  pareto  optimal.  Magnanti  and  Wong  (1981)  showed  that  pareto 
optimal  cuts  can  accelerate  the  convergence  of  the  cutting  plane  algorithm, 
particularly  for  Benders  decomposition.  Selecting  an  allowable  cut  or  a 
pareto  optimal  yk  would  require  considerable  effort  in  general.  Hearn  and 
Lawphongpanich  (1989a  &  b)  described  a  heuristic  method  for  selecting  an 
allowable  cut.  The  method  consists  of  the  following  two  rules: 

i)     If  fmax  >  1/  set  tk  =  fmax  -  e,  and 

ii)  If  fmax<  1,  set  tk  =  tmax  +  e 
where  £  is  a  small  positive  number.  Figure  2  illustrates  how  these  rules  select 
a  cut.  The  function  L(u)  is  nondifferentiable  at  vmax  and  hiax  <  1  in  Figures 
2(a)  and  (b).  So,  letting  tk  =  tmax  +  £  would  set  vk  to  the  right  of  vmax  by  an  e 
amount.  However,  at  vk  the  function  L(it)  is  differentiable  and  the  the  cut 
generated  here  is  the  line  tangential  to  L(u).  In  Figure  2(a),  this  e-purturbation 
rule  chooses  the  only  nondominated  cut.  In  2(b),  there  are  an  infinite  number 
of  nondominated  cuts  and  all  of  which  are  convex  combinations  of  the  two 
'extreme'  cuts.  In  this  case,  the  rule  selects  one  of  the  extreme  cuts. 
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previous  cuts 
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allowable  cuts 
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the  pareto  optimal  cut 
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previous 
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two  extreme 
cuts 

the  chosen 
cut 


Figure  2.  A  Heuristic  Section  of  Allowable  Cut 
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4.2  Difficult  Subproblems 

In  this  situation,  one  alternative  is  to  solve  the  subproblem  only  approxi- 
mately while  ensuring  that  effective  cuts  are  still  being  generated.  In 
Lagrangian  relaxation  or  subgradient  optimization  (see,  e.g.,  Fisher,  1981  and 
Polyak,  1969),  the  optimal  objective  function  value  is  used  to  define  the  step 
length  calculation  which  guarantees  convergence.  For  the  CP  algorithm,  the 
maximin  value,  w* ,  analogously  defines  how  close  the  approximation  need  to 
be  in  order  to  generate  an  effective  cut.  In  general,  the  value  zv*  is  unknown  at 
the  start  of  the  algorithm  and  must  be  estimated  to  make  the  resulting 
algorithm  effective.  However,  w*  =  0  for  VI  problems  (see,  e.g.,  Zuhovickii  et 
al.  1969  and  Auslender  1976).  This  allows  an  approximate  subproblem 
solution  to  be  easily  obtained. 

Below,  we  state  a  version  of  the  basic  CP  algorithm  which  solve  the  sub- 
problem  approximately.  The  name  'nontangential'  is  due  to  the  fact  that,  by 
approximately  solving  the  subproblem,  the  generated  cuts  are  not  necessarily 
tangential  to  Liu). 

The  Nontangential  Cutting  Plane  Algorithm  (NTCP) 

Step  0:   same  as  before. 
Step  1:   same  as  before. 

Step  2:   If  zvk=  w* ,  then  there  exists  an  optimal  solution  (zvk,uk)  such  that 
L(n*9  =  w*-  Otherwise  (i.e.,  w*  <  zvk),  select  xk  e  X  such  that 

f(xk)  +  ukg(xk)  <  w*.  (9) 

Set  k  =  k  +  l  and  go  to  Step  1. 
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Note  that  xk  in  Step  2  need  not  be  an  optimal  solution  to  the  subproblem.  In 
fact,  the  (subproblem)  objective  function  value  at  xk  only  need  to  be 
sufficiently  small,  i.e.,  no  larger  than  w*.  This  insures  that  (zuk,uk)  is  infeasible 
to  the  (k+lf*  master  problem  because 

f(xk)  +  ukg(xk)  <  zv*  <  wk. 
From  this  inequality,  it  can  be  concluded  that  wk  is  still  a  monotonically 
decreasing  sequence.  However,  because  xk  does  not  necessarily  solve  the  sub- 
problem,  the  cut:  f(xk)  +  ukg{xk)  is  not  necessarily  tangent  to  L{u)  at  uk  (see 
Figure  3).  Nevertheless,  the  following  results  show  that  NTCP  can  still 
converges  to  the  desired  solution. 


w 


w    -  - 


f(xk)  +  ukg(xk) 


previous 
cuts 


nontangential 


cut  at  u 


m 


Figure  3.  An  Example  of  a  Nontangential  Cut 
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Theorem  8:  \iwk  =  w* ,  then  there  exists  an  optimal  solution  the  the  master 


problem,  (zuk,uk),  such  that  L(uk)  =  w*.  That  is,  (x  ,u  )  solves  the 

maximin  problem,  where 

xk  =  min{f(x)+ukg(x)}. 

X  c  A 

Proof:  Assume  that  the  theorem  is  not  true.  Then,  for  every  pair  (zvk,uk)  which 
is  optimal  to  the  k^  master  problem,  the  following  must  hold 

L(uk)  <mir\{f(xi)  +  ukg(xi):i  =  0,...,(k-  1)}  =  wk  =  w*.  (10) 

However,  for  any  pair  (w,u)  feasible,  but  nonoptimal,  to  the  kth  master  prob- 
lem, the  following  also  holds 

L(u)  <  min{/(.t')  +  ug(xl):i  =  0,...  ,(Jt  -  1)}  <  w  <  wk  =  w*  (11) 

Combining  (10)  and  (11),  we  get 

L(u)  <w*      for  all  u  6  U 

or  maxL(K)  <  w*. 

ueU 

This  is  a  contradiction  since  w*  is  the  maximin  value.  □ 

Lemma  9:  Assume  that  NTCP  generates  an  infinite  sequence  {uk}.  If  X  is  com- 
pact and  there  exists  a  subset  K  £  {1,  2,  ...  }  such  that  the  subse- 
quence {i^'h-eK  is  convergent,  say  to  the  limit  point,  u°°,  then 

XV*  =  \imzu 
keK 

Proof:  Using  the  same  argument  as  in  Lemma  2,  it  can  be  shown  that  for  k 
sufficiently  large  and  k  €  K 
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w*  <ir  <  f(xk)  +  u°°g(xk)  <f(xk)  +  ukg(xk)  +  e.  (12) 

However,  by  the  choice  of  xk  in  Step  2  of  NTCP 

f{xk)  +  ukg{xk)<w*  (13) 

Combining  (12)  and  (13)  yields  the  following 

w*  <  w°°<f(xk)  +  u°°g(xk)  <f(xk)  +  ukg(xk)  +  e<  w*  +  z.  (14) 

Since  e  is  arbitrary,  we  can  conclude  that 

.* 


w*  =  lirmi; 
keK 


To  address  the  convergence  of  [u   }K  to  u*,  where 


w*  =  min{f(x)  +  u*  g(x)}, 
xex 


define  the  following 

i)       X"       =  {x'.x'.x2,...} 

ii)     [  X°°        =  the  closure  of  X' 


iii)    L(u)      =  r     min         lf(xl)  +  ugtf)\ 

I  €{  0,1,...,  (it- 1)  }  L  J 


iv)    L°°(u)      =  min   {f(x)  +  ug(x)} 

x€  X°°] 


From  (iii)  and  (iv),  it  is  clear  that 


L°°(tO<L*+1(iO<Lfc(tO  Vwetf, 


and  it  can  be  shown  that 


lim  Lk(u)  =  L°°(u)     Vu  €  U. 
lt-»  °° 


D 
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In  other  words,  [L(u  )}  is  a  monotonic  sequence  of  functions  which  converges 

pointwise  to  L°°(u).  Furthermore,  note  that 

k  k 

u    =  argmax  L  (u). 

u  £U 

k  oo 

If  the  subsequence  {u  }K  converges  to  u   ,  then  it  follows  from  Theorem  3.7  of 
Wets  (1983)  [see  also  the  results  in  Wets,  1980]  that 

u     =  arg  max  L°°(u), 
u  all 


and  (14)  also  implies  that 


w*  =  1^(11°°)  =  lim  w   . 
keK 


Lemma  10:  Under  the  same  assumptions  as  in  Lemma  9,  u*  solves  the 
following  problem: 

oo 

max  L  («). 

ueU 

Proof:  Note  that 

W*  =  min  {f(x)  +  i/  *  g(x)}        <     min     {f(x)  +  u*  g(x)} 
^•^  xe|xc  ' 


<  min        l{f{xi)  +  u*g(xij\ 

ie{o,i,...,(Jt-i)}  l  J 

<wk 

where  the  first  inequality  follows  from  the  fact  that  [X   ]  is  a  subset  of  X,  the 

second  from  the  fact  that  {0,1,2,.. .,(/:-!)}  is  a  subset  of  [X  ],  and  the  last  from  the 

th 

fact  that  u*  does  not  necessarily  solve  the  k    master  problem. 

The  above  sequence  of  inequalities  can  be  summarized  as  follows 

w*<  Lc°(ii*)<xuk. 
By  taking  the  limit  as  k  —■  °°  ,  k  €  K  ,  and  invoking  Lemma  9,  we  have  that 
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L   (u*)  =  w*  =  max  L   (n) 
u  eU 

and  the  proof  is  complete. □ 

An  immediate  consequence  of  Lemma  10  is  that  if  the  problem 

max  lT(u) 
uall 

k 
admits  a  unique  solution,  then  {u  }K  must  converge  to  u*.  Given  Lemma  9  and 

10,  the  convergence  of  NTCP  for  the  three  instances  of  maximin  problem  can 
be  established  as  in  the  case  of  the  basic  CP  algorithm  except  for  Benders 
decomposition.  To  insure  that  the  NTCP  algorithm  for  Benders 
decomposition  converges  finitely,  the  xk  chosen  in  Step  2  of  NTCP  must  also 
be  an  extreme  point  of  the  region  X. 

It  is  interesting  to  specialize  NTCP  to  VI  problems.  Because  of  the 
structure  of  Vis,  the  task  of  selecting  an  xk  in  Step  2  is  much  easier  and  the 
stopping  rule  and  the  convergence  result  can  both  be  strengthened.  Recall 
that  the  maximin  problem  for  Vis  takes  the  following  form 

to*  =  max  min  {F[x)x  -  F(x)u} 

where  S  is  nonempty,  convex  and  compact  subset  of  Rn  and  F(x)  is  a  vector 
valued  function  mapping  Rn  into  Rn.  Moreover,  F(x)  is  further  assumed  to  be 
strongly  monotone  which  implies  that  there  is  a  unique  solution,  u* ' ,  to  the  VI 
problem,  i.e., 

F(u*)(x  -  u*)  >  0  VxeS. 

Below,  we  state  the  version  of  NTCP  algorithm  for  Vis. 
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The  Cutting  Plane  Algorithm  for  Variational  Inequalities  (CPVI) 

Step  0:   Let  x°  e  S.  Set  it  =  1  and  go  to  Step  1. 
Step  1:   Solve  the  (k^)  master  problem 

max  w 

s.t.  w  <  Fix^x1-  F(rO«  i  =  0, ...,  (fc-1) 

it  €  S 

Let  (zvk,uk)  be  an  optimal  solution  and  go  to  Step  2. 
Step  2:   If  wk  =  0,  stop  and  a:' for  some  i  e  (0,  ...,  (k-1)}  is  a  solution.   Otherwise, 
select  xk  €  S  such  that 

F(xk)(xk-uk)<0. 

Set  k  =  k+1  and  go  to  Step  1. 

The  stopping  rule  in  Step  2  follows  from  the  fact  that  zo*  -  0  for  VI 
problems  (see,  e.g.,  Zuhovickii  et  al.  1969  and  Auslender  1976).  Theorem  11 
and  12  below  describe  how  to  obtain  the  solution  to  the  VI  when  CPVI 
terminates  finitely  and  when  it  generates  an  infinite  sequence,  respectively. 

Theorem  11:  If  zok  -  0  and  F(xJ)(x'-uk)  =  0,  then  x*  solves  the  VI  problem. 
Proof:    Under  the  strong  monotonicity  assumption,  the  solution  to  the  VI 
problem  must  be  unique  [see,  e.g.,  Auslender  (1976)  and  Hearn  et  al.  (1984)]. 
Thus,  there  must  be  only  one  x1  such  that 

Q  =  wk  =  F{x){xi-uk) 

and  for  i  ±  j 

0  =  w k  <F(x')(x'-uk). 
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Then,  the  A:th  master  problem  can  be  reduced  to  a  problem  with  only  one  cut, 

i.e., 

0  =  wk  =  max    w 

s.t.    w  <  F(x)  {x'-u) 
u  e  S, 

or  0  =  w    =  max  f[x,)(xj  -  u). 


However, 


0  =  max  F(xj)(xj  -it)     >    F{xj)(xj  -  u)  VueS 


Multiply  through  by  -1  to  obtain 

F(xj)[u  -  x1)  >0  VueS. 

That  is,  xJ  solves  the  VI  problem.  □ 

Theorem  12:  The  solution  to  the  VI  problem,  u*,  is  a  limit  point  of  the  set  [  X   J. 
Proof:  For  each  k,  let  i(k)  denote  an  index  of  an  active  cut  for  the  k     master 


problem,  i.e.,  i(k)  satisfies 


w    =  F(x  v  ')xK      -  F(x  v  ')u 


Then,  we  have 

2 


cc\xlW  -  11 


<  (F(xi{k))-F(u*))(xi{k)  -  u*) 

=       F(xi{k))(xi(k)  -  u*)  -  F(u*)(xi{k)  -  u*) 

<  F(xi{k))(xi{k)  -u*) 


wk 


The  first  inequality  follows  from  the  definition  of  strong  monotonicity  and  the 
second  from  the  fact  that  u*  solves  the  VI  problem,  i.e., 
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F(i?)(x-  u*)>0  VxeS. 

Since  S  is  compact,  there  must  exist  a  subset  K  —  {0,  1,  2,.  .  .  }  satisfying  the 
condition  in  Lemma  9,  thereby  having  the  property  that 

0  =  w*=  \imwk 
k£K 

Thus,  from  the  above  sequence  of  inequalities,  it  must  be  true  that 


lim 
keK 


x^  .  u 


2 

=  0    or    lim  x'{k)  =  u*. 
keK 


D 


i(kk 


From  the  above  theorem,  there  exists  a  subsequence  of  active  cuts,  [x  } , 
converging  to  it*.  Alternately,  Lemma  10  also  implies  that  the  sequence  {u  } 
also  converges  to  it*  when  it  'strongly'  solves  the  VI  problem  (see  Figure  4), 
i.e., 

F(n*)(x  -  u*)  >  0  Vx£S     &     x*u*. 

In  this  case,  the  problem:  max(L   (u)  :  u  e    S  }  admits  a  unique  solution.   To 
verify  this,  assume  that  «'  *  u*  is  an  alternate  solution  to  the  problem.   Then 

L°°(u')=     min     {F{x)(x  -  a')}  <F{u*\u*  -«*)  <  0. 

.r€[x°°] 

oo 

where  the  first  inequality  follows  from  the  fact  that  n*  £  [  X  ]  and  the  second 
from  the  fact  that  it*  'strongly'  solves  the  VI  problem.  However,  from  Lemma 
10,  u*  also  solves  max{L  (u)  :  u  £  S  )  and  L  (u*)  -  w*  =  0.  Thus,  the  above 
inequalities  is  a  contradiction  since  L°°(n*)  =  L  (u')  =  0.  Therefore,  when  u* 
'strongly'  solves  the  VI  problem,  the  entire  sequence  {  u  }  must  converge  to  u*, 
since  every  convergent  subsequence  of  {it  }  must  converge  to  a  solution  of  the 
problem  max{L  (it)  :  it  £  S  }  which  has  a  unique  solution  (see,  page  234  of 
Bazaraa  and  Shetty,  1979). 
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u* 

S                                               ■'     .    ..     .in-iiw.^i: ..  ....  ,3k  hj;.:                                       1 

\                       |5                        / 

Figure  4:  A  'Strong'  Solution  to  a  Variational  Inequality  Problem 

The  rate  of  convergence  of  CPVI  depends  in  part  on  the  choice  for  xk  in 
Step  2.  The  choice  requiring  the  least  effort  is  due  to  Zuhovickii  et  al.  (1979); 
they  set  xk  =  uk.  Under  this  choice 

F(xk){xk  ■  uk)  =  F{uk)(uk  V)=0. 

Nguyen  and  Dupuis  (1984)  proposed  another  choice  of  xk.  They  let  xk  be  the 
solution  to  the  VI  problem  over  the  line  segment  joining  xk~l  and  uk,  denoted 
as  i[xk~^ ,  uk].  Thus,  x    €  £[xk-]  ,  uk]  and  satisfies 

F(xk){V-xk)>0  Vye^Vj. 

Since  uk  £  £[xk~l ,  nk], 
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F(xk)(uk  -xk)>0     or    F(xk){xk  -uk)<0. 

So,  Nguyen  and  Dupuis  choice  of  xk  satisfies  the  requirement  in  Step  2. 
Figure  5  illustrates  this  choice  of  xk.  In  Figure  5(a),  if  F(xk)  is  a  gradient  of  a 
function,  say,  f(x),  then  xk  would  be  the  unconstrained  minimizer  of  a  one- 
dimensional  optimization  problem.  In  Figure  5(b),  the  'unconstrained 
minimizer'  is  to  the  left  of  ak.  So,  x  must  be  set  equal  to  uk.  However,  this 
suggests  a  generalization  of  the  choice  for  xk  by  Nguyen  &  Dupuis. 
Let 

aup  =  max \a:a  >  0  and  x  *   +  a[u    -  x  '  )  e  S\ 
and  set 

fc-1     .  (    k  k-\\ 

vup  =x       +aup(w    -x      ). 

Then,  choose  xk  €  £[  vup  ,  xfc_1  ]  such  that 

F(xk)(y-xk)>0     Vy  el[vuprxk-1}. 

Since  uk  e  (.[  vup  ,  xk~l  ], 

F{xk){uk  -xk)>0     or    F(x*)(x*  -  uk)  <  0. 

Thus,  the  new  choice  of  x^  also  satisfies  the  requirement  in  Step  2.   Figure  6 
illustrates  the  new  choice  of  xk. 
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Figure  5.   Nguyen  &  Dupuis's  Choice  for  xk 
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Figure  6.  The  New  Choice  for  xk 
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5.     CONCLUSION 

This  paper  addresses  the  cutting  plane  algorithm  which  has  been  pro- 
posed for  problems  derived  from  two  different  but  related  areas  in  operations 
research:  optimization  and  equilibrium  problems.  In  optimization,  the  cut- 
ting plane  algorithm  is  often  used  to  solve  the  master  problem  from  Benders 
decomposition  or  Lagrangian  dual  of  a  nonlinear  program.  For  the  equi- 
librium problem,  the  algorithm  has  been  applied  only  to  finite  dimensional 
variational  inequalities.  To  unify  ideas  in  the  two  areas,  this  paper  views 
problems  in  both  areas  as  maximin  problems.  This  establishes  a  common 
framework  for  analyzing  and  examining  convergence  properties  of  various 
schemes  for  enhancing  the  cutting  plane  algorithm.  In  particular,  the  analysis 
also  leads  to  several  interesting  new  results.  First  is  the  relationship  between 
adding  a  line  search  step  and  generating  'deep'  or  pareto  optimal  cuts. 
Second  are  the  concept  of  generating  nontangential  cuts  and  the  justification 
for  solving  the  subproblem  approximately.  The  last  are  an  alternate  proof  of 
convergence  and  a  generalization  of  an  existing  method  for  generating  cuts  in 
the  case  of  variational  inequalities. 

It  has  been  well  demonstrated  that  the  acceleration  strategies  discussed 
herein  actually  reduced  the  computing  time.  Hearn  and  Lawphongpanich 
(1989a  &  b)  compared  the  CP  algorithm  against  CPLS  on  25  Linear  and  25 
quadratic  integer  programming  problems.  The  number  of  variables  in  these 
problem  ranges  from  20  to  100  with  number  of  constraints  ranges  from  10  to 
50.  They  concluded  that  the  addition  of  line  search  saves  cpu  time  on  the 
average  40%  for  linear  problems  and  70%  for  quadratic.  The  rather  large 
saving  for  the  quadratic  case  is  due  to  the  fact  that  the  line  search  only  require 
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two  dual  function  evaluations.  In  Hearn  and  Lawphongpanich  (1990),  the 
same  comparison  was  made  on  a  traffic  assignment  problem  from  Goffin 
(1987)  with  22  arcs,  14  nodes,  and  23  origin-destination  pairs.  The  saving  due 
to  line  search  is  approximately  50%. 

As  for  variational  inequalities,  its  subproblem  is  generally  a  nonlinear 
program.  Thus,  by  entirely  eliminating  the  subproblem  from  the  steps  of  the 
CP  algorithm,  CPVI  can  only  accelerate  the  basic  algorithm.  To  demonstrate 
that  CPVI  is  competitive  with  existing  methods  for  variational  inequalities, 
Nguyen  and  Dupuis  (1984)  compared  CPVI  with  x  -  v*  against  the  well- 
regarded  Frank-Wolfe  algorithm  (see,  Frank  and  Wolfe,  1956  and  LeBlanc  et 
al.,  1975)  for  the  traffic  assignment  problem.  They  showed  that  CPVI  with  x  = 
v*  compares  favorably  with  Frank-Wolfe  on  problems  with  19  arcs,  11  nodes 
and  4  origin-destination  pairs  to  problems  with  2836  arcs,  1052  nodes  and  147 
origin-destination  pairs.    Also  shown  is  the  fact  that  CPVI  with  x    =  v*  is 

k  k  k 

superior  to  the  one  with  x    -  u    .  This  is  not  unexpected  since  setting  x    =  v*  is 

k         k-1 

tantamount  to  performing  a  line  search  along  the  direction  u  -  x  (see 
Hammond,  1984). 
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